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SCHUBERT VARIETIES OF TYPE A 
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Abstract. Schubert varieties in finite dimensional flag manifolds G/P are a well-studied family of 
projective varieties indexed by elements of the corresponding Weyl group W. In particular, there 
are many tests for smoothness and rational smoothness of these varieties. One key result due to 
Lakshmibai-Sandhya is that in type A the smooth Schubert varieties are precisely those that are 
indexed by permutations that avoid the patterns 4231 and 3412. Recently, there has been a flurry of 
research related to the infinite dimensional analogs of flag manifolds corresponding with G being a 
Kac-Moody group and W being an affine Weyl group or parabolic quotient. In this paper we study 
the case when W is the affine Weyl group of type A or the affine permutations. We develop the notion 
of pattern avoidance for affine permutations. Our main result is a characterization of the rationally 
smooth Schubert varieties corresponding to affine permutations in terms of the patterns 4231 and 
3412 and the twisted spiral permutations. 



1. Introduction 

The study of Schubert varieties and their singular loci incorporates tools from algebraic geometry, 
representation theory and combinatorics. One celebrated result in this area due to Lakshmibai- 
Sandhya is that in classical type A the smooth Schubert varieties are precisely those that are indexed 



by permutations that avoid the patterns 4231 and 3412 [241 ] . see also [3 ll: l3o|. A second important 
theorem in this area concerns a weaker notion than smoothness based on cohomology, called rational 
smoothness. In general, smoothness implies rational smoothness, but not conversely. For classical 
Schubert varieties, Peterson showed smoothness is equivalent to rational smoothness precisely in 



types A, D, E. Recent 



[ll; |2l|; |23|; |25|; |2J; [27 



y, there has been a surge of research activity related to affine Schubert varieties 



29l | . It is natural to ask how properties of smoothness, rational smoothness, 
singular loci and tangent spaces for affine Schubert varieties relate to their classical counterparts. 

In this paper we give a criterion for detecting rationally smooth Schubert varieties in affine type A. 
These varieties are indexed by the set of affine permutations, denoted S n . Generalizing the theorem 
of Lakshmibai-Sandhya, we show that the patterns 4231 and 3412 can be interpreted as patterns 
for affine permutations. A permutation avoiding these two patterns will again index a rationally 
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smooth affine Schubert variety. However, there is an infinite family of affine permutations in S n 
which contain 3412 and yet index rationally smooth affine Schubert varieties. These varieties are 
related to the spiral varieties studied by Mitchell 



30f ] . see also [lj]. Thus, our main result is a complete 



characterization of the rationally smooth Schubert varieties in G/B in type A n . 

Theorem 1.1. Let w 6 S n for n > 3. The Schubert variety X w is rationally smooth if and only if 
one of the following hold: 

(1) w avoids the patterns 3412 and 4231, 

(2) w is a twisted spiral permutation (defined in SectionW. 



Note, S2 is the infinite dihedral group. It follows that X w is rationally smooth for all w £ Sa- 
lience, throughout this paper we will assume n > 3 unless otherwise specified. 

Every point in X w which is not rationally smooth must be singular, hence we get a necessary 
condition to detect singular affine Schubert varieties. 

Corollary 1.2. Let w € S n for n > 3. If w contains either a 34-12 or 4231 pattern, then X w is 
singular. 

It was shown in [9] that there are only a finite number of 3412 avoiding affine permutations in S n . 
Thus, there exists a finite number of smooth Schubert varieties indexed by S n . Using this fact, we 
have verified the following conjecture up to n = 5. 

Conjecture 1. Let w £ S n . The Schubert variety X w is smooth if and only if w avoids 3412 and 
4231. 

This paper is organized as follows. In Section [21 we present the basic definitions and constructions 
needed in this paper. In Section [21 we begin the proof of Theorem ll.il by first discussing the case that 



w avoids the two patterns. In particular, we show that the Poincare polynomial of the cohomo 
ring of X w factors into palindromic factors as Gasharov showed in the classical type A case Q; 
By a theorem of Carrel-Peterson [8f], this implies X w is rationally smooth. In Section U we prove 
that if w contains the pattern 4231, then the Poincare polynomial fails to be palindromic at rank 1. 
In Section [5] we develop the theory of affine Bruhat pictures, which will be used in Section [6] to show 
that if w contains the pattern 3412 then it is either a twisted spiral permutation or we can identify 
a point in X w which is not rationally smooth. This will complete the proof of Theorem 11.11 We 
conjecture the rationally singular points obtained in the proof are maximal singular points. Finally, 
in Section [7] we discuss some directions for further research. 



rationally smooth affine schubert varieties 
2. Background and Notation 
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In this section, we will begin by collecting common notation and background necessary to prove 
the main theorem. We will proceed from general facts about Schubert varieties and Coxeter groups 
to specifics of the particular Coxeter group S n . 

2.1. Schubert Varieties. Let G be a semisimple Lie group such as GL n (C). Let B be a Borel 
subgroup of G. For GL n , we can take B to be the upper triangular matrices. The cosets G/B 
form the points of a flag variety. For each G, there is an associated finite Weyl group W. For each 
w 6 W, we obtain the Schubert variety X w by taking the Zariski closure of the orbit B ■ e w , where 
e™ represents w embedded in G. There are many good references on Schubert varieties including 

BHQ; 0; 0; 

For each semisimple Lie group G, there is an associated Kac-Moody group G and an associated 
affine Weyl group W obtained from W by adding one additional generator so- Weyl groups and 
affine Weyl groups are special cases of Coxeter groups. For G = GL n , the associated G is the set 
of invertible matrices with entries in C((z)), the field of rational functions of z. The affine analog 
of the flag variety is G/B, where B is the set of all invertible matrices, g, with entries in C[[z]], 
the formal power series in z, with the property that g\ z=0 is upper triangular. The affine Schubert 
varieties are the Zariski closures of the orbits B ■ e w for w in W. We use the same notation X w for 
Schubert varieties and affine Schubert varieties, but the index comes from an affine Weyl group if 
X w is an affine Schubert variety. For background on Kac-Moody groups and affine Schubert varieties 
we recommend (221 ]. 



2.2. Coxeter Groups. For a general reference on Coxeter groups, see 0] or [lij. Let W be a 
Coxeter group generated by a finite set S with relations of the form (siSj) mij , where each ma = 1 
and rriij > 2 otherwise. Any w £ W can be written as a product of elements from S in infinitely 
many ways. Every such product will be called an expression for w. Any expression of minimal 
length will be called a reduced expression, and the number of letters in such an expression will be 
denoted £(w), the length of w. Call any element of S a simple reflection and any element conjugate 
to a simple reflection, a reflection. There is a partial order on any Coxeter group W defined as the 
transitive closure of the covering relations, where u < v if there exists a reflection t with ut = v and 
£(u) = £(v) — 1. This partial order is called the Bruhat order on W . Under Bruhat order, W is a 
ranked poset, ranked by the length function. 

Given two elements u < w in W related by Bruhat order, let [u, w] = {v £ W : u < v < w} denote 
the interval between u and w. Call the interval [0, w] the order ideal of w, where u = is the identity 
element of W. The Poincare polynomial P w (q) is the rank generating function for the order ideal of 
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w. Specifically, 

This polynomial gets its name from the fact that P w (q 2 ) is the Poincare polynomial of the cohomology 
ring of X w . Note that v < w if and only if v^ 1 < w , so we have P w (q) = P w -i(q). 

We use the following theorem due to Carrell-Peterson to define rational smoothness in terms of 
the combinatorics of the Bruhat order and Coxeter groups. Let T be the set of all reflections in W, 
and let Ji(x,w) = {t G T : x < xt < w}. For any polynomial f(q) of degree n, call / palindromic if 
f(q)=t n f(q- 1 ). 

Proposition 2.1. [8, Theorem E] Let W be an (affine) Weyl group. Let w G W. Then the following 
are equivalent. 

(1) The (affine) Schubert variety X w is rationally smooth. 

(2) P w (q) is palindromic. 

(3) #R(x,w) = £(w) - £{x) for all x<w. 

The main goal of this paper is to determine for which w G S n , P w {q) is palindromic. Each w with 
a palindromic Poincare polynomial will factor using a parabolic decomposition as follows. Given a 
subset J C S of the generators for a Coxeter group W, we can define Wj to be the subgroup of W 
generated by the elements of J. Wj is called the parabolic subgroup generated by J. Each coset in 
W/Wj contains a unique element of minimal length [19., Proposition 1.10]. The set of all minimal 
length coset representatives is denoted 

W J = {w G W\e(ws) > £(w) for all s G J}. 

Thus, we often identify W J with these cosets. We will also need to use left cosets of Wj. Let J W 
denote the minimal length left coset representatives in Wj\W. Hence, 

J W = {w G W\£{sw) > l(w) for all s G J}. 

Bruhat order on W induces partial orders on J W and W J . When we wish to refer to the Poincare 
polynomial of a minimal length coset representative w for the induced order on either of the quotients 
J W or W J , we will denote it by ' P w or P^, respectively. The parabolic decomposition for elements 
of W is given as follows. 

Proposition 2.2. [5, Proposition 2.4.4,2.5.1] For every w G W there exists a unique u G Wj and 
a unique v G J W such that w = u ■ v and l(w) = £(u) + £(v) . Moreover, the map w i— > v is order 
preserving as a map from W to the set of minimal length coset representatives. 
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Lemma 2.3. 0, Lemma 2.4.3] An element w G W will be a minimal length coset representative 
for Wjw if and only if no reduced expression for w begins with an element of J. In the case where 
J = S\{s{} and w is not the identity, this implies every reduced expression for w must begin with Sj. 

Definition 2.4. Let w = s^ ■ ■ ■ Sj be any reduced expression for w G W. Let m(w, J) denote the 
element obtained by the following greedy algorithm. Initially set m to be the identity element. Then, 
reading left to right, inductively set m to be msi q whenever si q is in J and £(msi q ) > £(m). Finally, 
set m(uu, J) = m. 



Computing m(w, J) is equivalent to taking the Demazure product from [2CJ, Definition 3.1] of the 
subword of • • • Si p consisting of all of the letters from J. 

Proposition 2.5. Theorem 2.2] For every w G W and parabolic subgroup Wj C W , the ele- 
ment m(w,J) is the unique maximal element in [0,w] Cl Wj. In particular, m(w,J) is well-defined, 
independent of the choice of reduced expression for w . 

Proposition 2.6. [2, Theorem 6.4] Suppose w G W has the parabolic decomposition w = u • v with 
u eWj and v G J W. If u = m(w, J), then P w (q) = P u {q) • J Pv{q)- 

2.3. Affine Permutations. Let S n denote the set of all bijections taking [n] = {1, 2, . . . , n} to itself. 
We denote w G S n by its one-line notation w = [wi,W2, ■ ■ ■ ,w n ] if w maps i to W{. As a Coxeter 
group, S n is generated by the adjacent transpositions Sj interchanging i with i + 1 with relations 
SiSj = SjSi for \i — j\ > 2 and SjSj+iSj = Sj+iSjSj+i for all 1 < i < n — 2. Note, S n is the Weyl group 
of type A that corresponds with the Lie group G = GL n (C). 

Let S n denote of the set of all bijections w : Z — > Z with w(i + n) = w(i) + n for all % G Z and 
Y^=i w {i) = ("J )■ i s called the affine symmetric group, and the elements of S n are called affine 
permutations. This definition of affine permutations appeared in [28:, §3.6] and was then developed 



m 



32]. 



We can view an affine permutation in its one-line notation as the infinite string 

• ■ • w^iw wiw 2 ■ ■ ■ w n w n+ i 

where Wi = w(i). An affine permutation is completely determined by its action on any window of 
n consecutive indices [wi, Wi+i, . . . ,Wi +ri ^\]. In particular, it is enough to record the base window 
[u>i, . . . ,w n ] to capture all the information about w. Sometimes however, it will be useful to write 
down a larger section of the one- line notation or use a different window. 

We can also view an affine permutation as a matrix. As mentioned in Section 12.11 the entries 
of the matrix will come from C((z)). For any w = [wi, . . . ,w n ] G S n , write Wi = aj + nbi, where 
1 < en < n for each 1 < i < n. Then w := [a\, . . . , a n ] G S n and ^ 6j = 0. We will define the affine 
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permutation matrix for w to be the n x n matrix M = (rriij) with j = and all other entries 
equal to 0. In the case where w = W G 5 n , this reduces to the usual notion of a permutation matrix. 
Given i ^ j mod n, let denote the affine transposition that interchanges i and j and leaves all 



k not congruent to i or j fixed. Since tij = t 



i +n j +n , it suffices to assume 1 < i < n and i < j. Note 
that if 1 < i < j < n, the above notion of transposition is the same as for the symmetric group. 
As a Coxeter group, S n is the affine Weyl group of type A. It is generated by 

S = {si := t i)i+1 : 1 < i < n - 1} U {s := t n , n+1 }. 

We will denote the set of all affine transpositions by 

T = {tij : 1 < i < n,i < j,i ^ j mod n}. 

The affine transpositions are the reflections in S n . The length function of S n and S n can be described 
in terms of one-line notation. For a permutation w € S n one can define an inversion in w as a pair 
such that i < j and iOj > Wj. The length of w G 5 n is the number of inversions in w. For 
an affine permutation, if u>j > Wj for some i < j, then we also have Wi+ n > Wj+ n . Hence, any 
affine permutation with a single inversion has infinitely many inversions. Thus, we standardize each 
inversion as follows. Define an affine inversion as a pair such that 1 < i < n, i < j, and 

wi > Wj. Let Inv?r (w) denote the set of all such affine inversions. Then we have £(w) = #Invs (w), 
[a, Proposition 8.3.1]. There is also another characterization of the length of an affine permutation 
due to Shi, which we won't need explicitly, but we include for context. 



Proposition 2.7. 



32, Lemma 4.2.2] Let w € S„. Then 



where invfioi, 



iw 



,W r , 



E 

l<i<j<n 



n 



mv 



{wi,...,w n )+ 

l<i<j<n 



\Wj — Wi\ 

n 



#{1 < i < j < n : Wi > Wj}. 



In terms of the one- line description of the affine symmetric group, we also have the following 
characterization of the covering relations for Bruhat order. 



26l . Lemma 2.2] Let w G S n and i < j. Then wtij < w provided 



Proposition 2.8. 

(1) Wi> Wj, 

(2) for all i < k < j, Wk £ [wj,Wi], and 

(3) either j — i < n or Wi — Wj < n. 



Define the rank function for w G S n by 



r w (p, q) = #{« < P ■ m > q}- 
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Define the difference function for the pair x,w E S n by 

d x ,w(p,q) = r w (p, q) -r x (p,q). 

The difference function gives another useful characterization of Bruhat order and generalizes the 



111 ], see also 



12 



Ehressmann criterion for Bruhat order 

Proposition 2.9. [f], Theorem 8.3.7] For x,w E S n , x < w if and only if d XtW (p,q) > for all 
p,q E Z. 

Since r w (p, q) = r w (p+kn, q+hn) for all k E Z, it suffices to check the inequalities in Proposition ^, 91 
for 1 < p < n and q E Z. Note that Proposition 12.91 is also valid if we replace r,,, and d x ^ w with 

?i0>9) = #{i > P ■ m < q} 

and 

d' x , w (p,w) =r' w (p,q) -r' x (p,q). 

Corollary 2.10. We have v < x < w if and only if d VtW (p, q) — d x<w {p, q) > for all p,q E Z. 

The following proposition characterizes the maximal proper parabolic subgroups in S n . Here we 
use the notation [a, b] := {i G Z : a < i < b} for intervals among the integers. 

Proposition 2.11. [5|, Proposition 8.3.4] Let i £ [n] and J = 5"\{sj}. T/ien 

(<S' n )j = Stab ([i, i + n—1}) 

and 

(S n ) J = (w G S n : u>i < ■ ■ ■ < Wi and < • • • < j . 

Corollary 2.12. If w G S n is not in any proper parabolic subgroup, then for every 1 < i < n and 
for every i < j < i + n, there exists some i < k < i + n such that either < Wj or Wk > Wj +n . 

2.4. Patterns in S n . Given a permutation p E and an affine permutation w £ §„, we say that w 
contains the pattern p if there exists any subsequence of integers i\ < ■ ■ ■ < ik such that the subword 
UJjj • • • Wi k of w has the same relative order as the elements of p. We say w avoids the pattern p if 
no such subsequence exists in w. For example, if w = [8, 1, 3, 5, 4, 0] E Sq, then 8,1,5,0 has the same 
relative order as the pattern 4231 so w contains 4231, however, w avoids 3412. Observe that a pattern 
can come from terms outside of the base window [w\, . . . ,w n ]. Moreover, the indices corresponding 
to terms in the pattern can have the same residue mod n. In the previous example, w also has 2,8,6 
as an occurrence of the pattern 132. Choosing a subword Wi 1 ■ ■ ■ Wi k with the same relative order as 
p will be referred to as placing p in w. 

We will need the following result, due to Lakshmibai and Sandhya. Note that in type A, smooth- 
ness is equivalent to rational smoothness for Schubert varieties lOi ] . 
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Proposition 2.13. [24, Theorem 1] A Schubert variety X w C GL n (C)/2? is (rationally) smooth if 
and only if w avoids the patterns 3412 and 4231. 

Corollary 2.14. Let J = S\{i} and suppose w G {S n )j. If w contains the pattern 4231 or 3412, 
then there is an occurrence of that pattern at indices i < i\ < i2 < i-s < 14 < i + n, so that w is not 
palindromic. 

Proof. Suppose the pattern, p, occurs at indices i\ < i<i < 23 < i± in w. By periodicity of w we may 
assume i < i\ < i+n. Since w G (S n )j, Proposition 12 . 1 1 1 implies {w{, . . . ,Wi +n -\} = {i, . . . ,i+n—l}. 
Hence, Wj > for all j > i + n. Since Wi 4 < , we must have i^ < i + n. Recall that (S n )j — S n . 
Since p is contained in the window [wi, . . . , U7j+ n _i], the image of w under this isomorphism will also 
contain p. Hence, by Proposition 12.11 and Proposition 12,131 w is not palindromic. □ 

2.5. Twisted Spiral Permutations. For a,b G Z, define 

S a S a+ iS a +2- ■ ■ Sa+b-l, if b > 0, 



(2.1) c(a,b) 



s a s a -ls a -2- ■ ■ Sa+b+l, if b < 0, 
1, if 6 = 0, 



where all of the subscripts are taken modulo n. By definition, we have c(a, 1) = c(a, —1) = s a , and 
£(c(a, b)) = \b\. For any k G Z with k 7^ and any 1 < % < n, let 

(2.2) w^ k) = w£c{i, k{n - 1)) £ S n , 

where Wq 1 is the unique longest element in the parabolic subgroup generated by J{ = S\{i}. For 
k 7^ 0, we call vj( l ' k ^ a twisted spiral permutation, since its reduced expression is obtained by spiraling 
around the Coxeter graph for S n and then twisting by Wq' . We could also define the twisted spiral 
permutations by using right cosets and twisting on the right. This definition is equivalent because 
of the formula 

(2.3) Woc(i, k(n - 1)) = c(i - kn ± 1, k(n - l))^"* , 

where we use + if and only if k > 0. Thus, the set of twisted spiral permutations is closed under 
taking inverses. 

By a result of Mitchell 30], the Poincare polynomial for c(i,k(n — 1)) in the quotient Ji W is 
palindromic. The quotient Ji W corresponds with the affine Grassmannian of type A. To be specific, 
we need the following definitions for q-factorial and q-binomial coefficients. For m G N, let 

q k , fk\ (kl) q 



1 k /u\ 
(2.4) ( m!)g = J|_^ and 



be the (/-analogues of ml and (^) 
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Proposition 2.15. 30, Prop. 2.4 and 3.2] Let J = S\{i} and let Wj be the corresponding maximal 
parabolic subgroup ofW = S n . Then, the Poincare for the Schubert variety X c uu n _]\\ in the affine 
Grassmannian is 



■*c(i,fc(n-l)) 



k + n — 1 

k 



q 

Proposition ^. 6l then shows that we may lift each c(i, k[n— 1)) to a palindromic element in W by left 
multiplying by Wq . Hence the Poincare polynomial for w^ l,k ^ is palindromic. However, each twisted 
spiral permutation contains the pattern 3412. We will see in Section O that elements of this infinite 
family, |u;(w : k G Z\{0}, 1 < i < n}, are the only palindromic permutations containing 3412. 

Remark 2.16. For k > 1, we note that X c (i,k(n-i)) ^ s n °t a smooth variety in the affine Grassman- 
nian by f^, Thm 1.1] since it is not a closed parabolic orbit. Hence, we see X (%,k) cannot be smooth 
in the affine flag variety G/B. 

3. Factoring Poincare Polynomials 

The goal of this section is to prove the first direction of the main theorem as stated below in 
Theorem 13.11 We will use an adaptation of the proof for the non-affine case given by Gasharov in 
In particular, we show that if w avoids 3412 and 4231, we may repeatedly factor off palindromic 
terms from P w (q). For w G S n , Gasharov shows that the factors of the Poincare polynomial for 
(rationally) smooth elements are all of the form (1 + q + ■ ■ ■ + q k ). However, for w G S n , we will see 
that the factors sometimes end up being the more general (/-binomial coefficients instead. 

Theorem 3.1. Let w G S n be an affine permutation that avoids the patterns 3412 and 4231. Then 
P w (q) is palindromic. 

Given any finite ordered sequence of integers t = t\t2 ■ ■ ■ t p , define the subsequence of left to right 
maxima of t, denoted a = a\02 ■ ■ ■ 0^, as follows. Set a\ = t\ and i\ = 1. Inductively, let ij + \ be 
the smallest index greater than ij with U j+1 > t^ for all k < ij+x- Then set Oj+i = ti j+x - Note that 
by its construction t\ = a\ < ■ ■ ■ < ag. This subsequence can be obtained by reading t from left to 
right and recording the value each time a new maximum is obtained. Similarly, we can define the 
subsequence of right to left maxima of t, denoted b = 6162 by reading t right to left. Note that 

by its construction b\ > ■ ■ ■ > bp = t p . 

Let w G S n and a G Z. Define the subword of a-inversions of w by v ^ = w^w^ ■ ■ ■ Wi k , where 
iq = a and i\ < ^2 < • • • < ik indexes the subsequence of all values to the right of a forming an 
inversion with w a . Call uninterrupted if is a consecutive subword of w. Otherwise, we say 
t>( a ) is interrupted. We say wp interrupts if a < (3 < i^ and wp > w a so that wp is not in v ^ . 

Throughout the rest of this section, assume w G S n and w is not the identity element. Fix 
a G Z to be the index such that w a is the largest element in the window [uui , . . . , w n ] that has a 
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non-trivial subword of a-inversions. Such an a exists since w is not the identity permutation. Let 
v = v ( a ) = WigWi-L ■ ■ ■ Wi p be the subword of a-inversions of w, and let u = ui ■ ■ ■ ut be the subsequence 
of right to left maxima in v. Let v\u denote all terms of v, which are not in u. Note u% = v\ = w a 
and ug = Wi p by construction. 

Next, we prove the afhne analogue of the main lemma Gasharov uses in his proof of the non-amne 
version of Theorem 13.11 

Lemma 3.2. If w avoids the patterns 4231 and 3412 and v is not decreasing (and hence not equal 
to u), then u = w a (w a — 1) • • • (w a — m) for some m. 

Proof. Since w avoids the pattern 4231, for any Wi E v\u we must have Wi < U£. To see this, suppose, 
par contradiction, that there exists an Wi E v\u with wi > U£. Since Wi £ u, there must exist some 
i < j < i p such that Wj E u and Wi < Wj. But then w a WiWjUi forms a 4231 pattern, giving a 
contradiction. 

Since there exists some Wj E v\u, which must have Wj < v,£ by the above argument. The 

definition of a guarantees that for any integer i < a, we have Wi < w a . Thus, for any i<awe must 
also have W{ < ug or else WiW a WjUn will form a 3412 pattern in w. 

Let m = w a — ue- Then by the previous argument and the fact that w : Z — > Z is a bijection, 
every value between w a and w a — m must occur in v, or else u = v. All of these values must occur 
in decreasing order, otherwise w would have a 4231 pattern. □ 

We now prove a lemma that is new to the affine case that arises since the subsequence v need not 
be a consecutive subsequence. 

Lemma 3.3. Suppose w avoids 3412 and 4231, and v is interrupted. If wp is the largest element 
of w that interrupts v, then wp starts a consecutive decreasing subword ending in ug in the one-line 
notation for w. 

Proof. Suppose that Wi interrupts v and that there exist two elements of v to the right of Wi not in 
decreasing order. Then there are Wj,Wk E v with i < j < k and wj < w^. Since Wi interrupts v we 
must have w a < W{. But then w a WiWjWk will form a 3412 pattern. Hence, all elements of v to the 
right of the first interrupting subsequence must occur in decreasing order. 

Let w r be an element in the consecutive subword lying between Vj-i and Vj, and let w s be an 
element in the consecutive subword lying between v m -i and v m . Suppose j < m so that r < s. If 
w r > w s , then, since v is decreasing to the right of the first interrupting subword, w r > w s > w a > 
Vj > v m . Thus, w r VjW s v m forms a 4231 pattern, giving a contradiction. Hence, every element in one 
interrupting subword is larger than any element in any previous interrupting subword. 

Let vug be the largest element of w interrupting v, and recall ug = Wi . By the above paragraph, 
W/3 must occur in the last interrupting subsequence. Then for any (3 < j < i p , we must have 
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Wj < W/3 since wp was chosen to be the maximal interrupter. We must also have Wj > u>i p , since v 
is decreasing to the right of the first interrupting subsequence. Then all Wj with j3 < j < i p must 
occur in decreasing order or else we get a 4231 pattern. □ 

Definition 3.4. Let x = x\X2 ■ ■ ■ be a consecutive subsequence of the one-line notation of w. Call 
x a factoring subword of w if it has all of the following properties: 

(1) x is decreasing, 

(2) x\ is larger than all elements to its left in w, 

(3) Xfc is smaller than all elements to its right in w, 

(4) if Xk is larger than all elements to the left of x\ in w, then all elements to the right of Xk in 
w are larger than x\. 

Note that Property (JTJ) implies k < n since there cannot be a consecutive decreasing subword of 
length longer than n in any element of S n . 

Lemma 3.5. Suppose w avoids the patterns 3412 and 4231. Then either w or w^ 1 (or both) has a 
factoring subword. 

Proof. If v is decreasing and not interrupted, set x = v , and x will be a factoring subword. To see 
that Property holds, note that since v is not interrupted, X\ = w a , and no element to the right 
of Xk in w can form an a-inversion. 

If v is not decreasing and not interrupted, then by Lemma 13.2^ u = w a {w a — 1) • • • (w a — m) for 
some m. Let x be the subword of w~ x corresponding to u. In other words, x = w~ a _ m ■ ■ ■ w^^w^K 
Note, x is a consecutive subword of w~ l , since the values of u are all consecutive. Property ([T]) holds, 
since u is decreasing. Since w a is larger than all elements to its left in w, will be smaller than 
all elements to its right in u> _1 , so that Property ^ holds. Similarly, since m is smaller than all 
elements to its right in w, w~ _ m will be larger than all elements to its left in w^ 1 . Thus, Property 
([3]) holds. Finally, if there exists some j > w a , such that a = < wj 1 < w~^_ m = i p , then this 
contradicts the hypothesis that v is not interrupted. Hence, Property (JH) is always true. Thus, x is 
a factoring subword of w" 1 . 

If v is interrupted, let be the consecutive subsequence of w starting at the maximal 

interrupter, wp, and ending at ug. By Lemma 1331 properties {I])-® hold. Since w a is not in x and 
Xk < w a , Property is vacuously true. Hence, x is a factoring subword. □ 

3.1. The Factoring Map. The goal of this subsection is to define a map : w i— > w' such that 
P w i{q) divides P w (q) and the quotient P w (q)/P w '{q) is palindromic. This map will be defined when- 
ever w avoids the pattern 3412 and has a factoring subword. 
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Define & as follows. Let be a factoring subword for w, and let Xj be the rightmost 

element in x such that there is no element of w to the left of x\ that is bigger than xj. Call Xj the 
pii>oi poini of the factoring subword. In the event where j = k, use X\ as the pivot point instead. 
This will happen, for example, when the subword of a-inversions is not interrupted. 

Suppose Xj = Wry is the pivot point embedded in w. Using the definition of c(a, b) in Section |2~5| 
set 

cr- 1 = 0(7, k - j)c(7 - 1, k -j) ■ ■ ■ 0(7 - (j - 1), fc - j). 

Define ^(it?) = w ■ cr" 1 , so that w = l F(w)a. In terms of the one-line notation for w, this has the 
effect of shuffling x\ ■ ■ ■ Xj to the right past Xj + \ ■ ■ ■ Xk- 
As an example, take n = 6 and 

w = [8,3,1,0,4,5)14,9,7,6, 10,11- ■■ e5 6 . 

Then w a = 8 so that v = 8,3,1,0,4,5,7,6 and u = 8, 7, 6. A factoring subword is x = 14,9,7,6, 
which has pivot point X2 = w% = 9. Hence, 

cr' 1 = c(8, 2)c(7, 2) = s 8 s 9 s 7 s 8 = S2S3S1S2, 

where the indices of the generators are taken mod n, so that 

&(w) = w ■ s 2 s 3 sis 2 = [1, 0, 8, 3, 4, 5] 7, 6, 14, 9, 10, 11 • • • . 



We collect here some facts about a and \P(w). Fix the notation d = 7 — (j — 1) + (A; — j — 1) and 
J = S'\{srf}. Then, value x\ occurs in position d + 1 in ^(w), and occurs in position d. Since 
k < n, a corresponds to a non-affine permutation in Wj with exactly one descent at position d. By 
Lemma 12.31 we immediately get the following lemma. 

Lemma 3.6. Every reduced expression for a begins with the letter s^, so that a £ J (S n ). 

Recall the definition of the g-binomial coefficients in (|2.4p . Then we have the following lemma. 

3 q 



Lemma 3.7. J P a {q) = ( ■) which is palindromic. 



Proof. Since a has only one descent, the order ideal of a in the quotient J (S n ) corresponds to the 
classical Grassmannian G(j, k) consisting of all partitions that fit inside a (k — j) x j rectangle. In 



34 . Proposition 1.3.19] it is shown that P a {o) = ( fc ) > which is known to be palindromic by box 
complementation. □ 

We now show that \P(w) always lies in some proper parabolic subgroup whenever it is defined. 
Note that in the proof below we only explicitly use avoidance of the pattern 3412. However, the 
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assumption that w avoids the pattern 4231 is implicitly used to guarantee the existence of a factoring 
subword in Lemma 13.51 

Lemma 3.8. Suppose w avoids the pattern 3412 and contains a factoring subword X — X \ X . Set 
w ' = &( w ). Then w' e {S n )j. 

Proof. Let Xj be the pivot point of x, so that w' d+1 = x\. Let w' s be the largest element strictly to 
the left of x\ in w' . We will show 

Wd-n+i, ■ ■ • > w' d ] = {w' s - n + 1, w' s - n + 2, . . . , w' s }. 

Any element in {w' d _ n+1 , . . . , w' d } is at most w' s , since w' s was chosen to be maximal. Suppose 
there is some m with < m < n — 1 and w' d _ m < w' s — n + 1. Then w d _ m < w' s — n, since both w' s 
and w' s _ n cannot be in the window {w' d _ n+1 , . . . , w' d }. Thus, w' d _ m + n = w' d _ m+n < w' s . 

First, suppose w' s is not in x, so that w' s > Xk and w' s lies to the left of x\ in w. If w' d _ rn is not 
a value in x, then w' d _ m lies to the left of Xj + \ in w' and w' d _ m+n lies to the right of x^ in w. So, 
w' d _ m+n > Xk by Property ([3]) in Definition 13.41 But then w' 8 xiXkw' d _ m+n forms a 3412 pattern in 
w, giving a contradiction. Otherwise, if w' d _ m is in x, then w' d _ m lies to the right of x\ in w, so that 
w 8 x i w d-m w d-m+n wu ^ f° rm a 3412 pattern in w, giving another contradiction. 

Second, suppose w' s is in x. By the construction of the pivot point, this can only happen if there are 
no elements to the left of x\ in w that are larger than x^. Then and X2 = w a+ i = w' s . 

Also, w' d _ m+n < X2 < w a , so that, by Property fl3J) in Definition 13.41 w' d _ m+n is in x. But then w^_ m 
lies to the left of x\ — n in w' , which is a contradiction, since w' d+1 = x\. □ 

Lemma 3.9. The element \P(w) = w' is maximal in (S n )j (~l [0, w]. 

Proof. Since w' G {S n )j and a € J S n , then to = u/cr is the parabolic decomposition for w given 
in Proposition 12.21 Let • • • Si be a reduced expression for w' , and let SdSj 2 ■ ■ ■ Sj r be a reduced 
expression for a. Then ■ ■ ■ Si q SdSj 2 ■ ■ ■ Sj r is a reduced expression for w. Use the construction of 
m(w, J) given in Definition 12.41 Since w' € (S n )j, start by choosing all of the letters in s^, . . . , Si . 
Then skip Sd, since it is not in J. Picking any letter of the reduced expression for a aside from Sd 
reduces the length of w', since the elements in Xj+i, . . . , x^ and x\, . . . , Xj occur in decreasing order 
in w' . Hence, m(w, J) = w' . □ 

Lemma 3.10. If w has a factoring subword and avoids the patterns 3412 and 4231, then \P(w) = w' 
also avoids 3412 and 4231. 

Proof. Let be a factoring subword for w, and let Xj UJ'y be the pivot point. Partition 

x as y = x\ ■ ■ ■ Xj and z = Xj + \ ■ ■ ■ Xk- Since x is decreasing, this implies every element of y is larger 
than every element of z. Moreover, when considered as subwords of either w or w 1 , every element of 
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y is larger than all elements to its left not in y. Consider the following consecutive sequences in w 
and w': 



w = • • • (y — n) (z — n) [ty z] (t + n) ■ ■ ■ , 
w' = ■ ■ ■ (z - n) (y — n)[tzy](t + n) ■ ■ ■ , 

where t is the (possibly empty) subsequence of indices between (z — n) and y in w so [tyz] forms a 
window of length n. If w 1 contains either of the patterns 4231 or 3412, we may assume the 4 of the 
pattern occurs in the window [t z y] . 

First, suppose w' has a 4231 pattern at indices i± < ii < i$ < i\. If w[ 4 occurs in t or z, then no 
other part of the 4231 pattern occurs in y, or any of its translates, since all values in y are bigger 
than all values to the left of y. Hence, the relative positions of the 4231 pattern remain unchanged 
in w, so that w also contains 4231, giving a contradiction. If w\ 4 occurs in y, then only the 2 of the 
4231 pattern may occur in y, since y is decreasing, and no other terms of the pattern may occur 
in any translate of y. This follows since every element of y is larger than all elements to the left 
of y. The remaining terms of the 4231 pattern occur in translates of t and z, so that the relative 
positions of the 4231 pattern remain unchanged in w. Hence, w also contains 4231, giving another 
contradiction. 

Second, suppose w' has a 3412 pattern at indices 13 < 14 < i\ < %%. If w\ occurs in t or z, then 
w'^ and w'i cannot occur in y. Hence, they still occur to the right of w' i4 in w and in the correct 
relative order. Since w', occurs to the left of wi in both w' and w, we get that w also contains 
3412. If w' i4 occurs in y, then we may assume w' i4 = x\, since y is decreasing. Since w'^ and w[ 2 
cannot occur in any translate of y, they still occur to the right of w\ 4 in w and in the correct relative 
order. If w' in is not in z, then «/ still occurs to the left of w',, in w, so that w also contains 3412. 
So suppose, in addition, that w' i3 occurs in z. Note that, by Property © in Definition 13.41 we must 
have Xk < w'^ < w' i% . Moreover, since w' is is in z, and z is decreasing, we must have w' i2 < Xj + \. If 
there exists a value w' m occurring to the left of Xj + \ in w' with Xj + \ < w' rn < x\, replace w' ia with 
w' m . Then w' m will occur to the left of w\ 4 in w and w' i2 < < w' m , so that w contains 3412. 

Note that such a value is guaranteed to exist whenever j > 1, by definition of the pivot point of x. 
If j = 1 and no such value exists, then by Property dU in Definition 13.41 w[ and w' i2 cannot exist, 
contradicting the assumption that w' contains 3412. □ 

Proof of Theorem \3.1[ The proof is by induction on £(w). Since w avoids the patterns 3412 and 4231, 
by Lemma 13.51 either w of w~ 1 has a factoring subword. We may assume w has a factoring subword 
since P w (q) = P w -i(q). Set w' = &(w), so that w = w'a. By Lemma [331 we have w' = m(w, J). By 
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Lemma [376], we have a G J (S n ). Hence, by Proposition 12 .6|. 

P w (q)=P w >(q)- J Pa(q). 

By Lemma 13.71 J P a (q) is palindromic (in fact, a (/-binomial coefficient). Lemma 13. 1UI shows that 
w' avoids 3412 and 4231, so by induction, P w i{q) is also palindromic. □ 

4. When w Contains 4231 
The goal of this section is to prove the following theorem. 

Theorem 4.1. Let w G S n be an affine permutation that contains the pattern 1^231. Then P w (q) is 
not palindromic. 

In particular, we show that if 

£(w) 

Pw{q) = ^c iq \ 
i=0 

then c\ < C£{ w )—i- Recall that c\ counts the number of unique generators occurring in any reduced 
expression for w, and that cm w \\ counts the number of affine permutations covered by w in Bruhat 
order. We will construct a graph whose edges correspond to these covering relations. Whenever w 
contains 4231, this graph will have more than c\ edges. 

Let w E S n . Fix (3 E Z such that W{ < wp for all i < f3, and let a\,...,a r be the indices 
corresponding to the subsequence of left to right maxima in the window [wp, . . . , wp+ n -i\. Let Gp 
be the graph on the vertices 

(4.1) Vp = {j G Z : j > p and Wj <w ar }, 
with edges 

(4.2) Ep = {(j, k)eVpxVp:P<j <P + n,j <k, and £(wt^ k ) = £(w) - 1} . 

For 1 < i < r — 1, let Hip be the induced subgraph of Gp on the vertices [o«, a^+i), and let H Tj p be 
the induced subgraph of Gp on Vp H [a r , (3 + n). Let Hp = H\ p U • • • U H T p. 

The following technical lemma is used to determine if w is in a proper parabolic subgroup or not. 

Lemma 4.2. If r > 2, and there exists some 1 < i < r — 1 such that for all Oj+i < b < a^i + n, 
w b > w ai , then w is in a proper parabolic subgroup. If w is not in a proper parabolic subgroup, then 
either r = 1, or for all 1 < i < r — 1 there exists some aj+i < j < a>i+i + n, such that Wj < w ai . 

Proof. Suppose r > 2 and such an i exists. Let Wj = min{ti; ai+1 , . . . , w ai+1 +n-i\- Note that Wj > w ai 
by our assumptions on i. Also, by our choice of j, for all a^+i < b < aj+i + n, we have Wj < Wb- 
Suppose there exists some Oj+i < k < ai + \ + n with > Wj + n. Then k — n < a^+i and 
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w k-n > Wj > w ai > w ai = wp. In particular, - n / Oj. If k — n < (3, then Wk- n > wp, 
which contradicts the choice of /3. If (3 < k — n < Oj, this contradicts Oj being a left to right 
maximum. Finally, if aj < k — n < Oj+i, this would mean there is some left to right maximum 
between etj and Oj+i, contradicting the fact that dj and Oj+i are two consecutive maxima. So we 
must have < w ai+1 , . . . , w; ai+1+n _i < tUj +n. Hence, by Proposition l2.11l w is in a proper parabolic 
subgroup. The second statement of the lemma is the contrapositive of the first. □ 

The next lemma will be useful for counting edges in Gp. 

Lemma 4.3. Let w £ S n , and let i < j be such that Wi > Wj. If either j — i < n or W{ — Wj < n, 
then there exists an increasing sequence i = i\ < ?2 < ■ ■ ■ < ik = j such that &(vuti m ,i m +i) = ^( w ) ~ 1 
for each 1 < m < k. 

Proof. Suppose j — i < n. Pick i2 such that > Wi 2 > wj and i<± — i\ is minimal. By Proposition 
12.81 £(wti lt i 2 ) = £(w) — 1. Similarly, if we have defined i\ < ii < ■ ■ ■ < i r -\ < j, then we can 
pick i r such that Wi r _ 1 > Wi r > Wj and i r — i r -\ is minimal. Then again by Proposition 12. 8\ 
(■{wtir-x^r) = £(w) — 1. The same proof works when Wi — Wj < n, only instead choose i r to minimize 
Wir-! -m r - 1=1 

Lemma 4.4. Each H^p is connected, and hence Hp has at least n — r edges. 

Proof. We start by showing that if ai < j < a^ + i for some 1 < i < r, then there is a path in Gp 
from dj to j. By the construction of the Oj, we always have aj + i — Oj < n, so that j — ai < n. Since 
w ai > Wj, such a path exists by Lemma 14.31 Similarly, if a r < j < (3 + n, then there is a path in 
Gp from a r to j. Hence, each H^p is connected. Since Hp is a graph on n vertices with at most r 
connected components, it has at least n — r edges. □ 

In the case where w is not in a proper parabolic subgroup, we can improve on the lower bound 
for the number of edges in Gp. 

Lemma 4.5. If w € S n is not contained in any proper parabolic subgroup, then Gp has at least n 
edges. 

Proof. First suppose r = 1. Then wp is the largest element in the window [wp, . . . ,wp +n -i]. Let 
Wj = minjii;^, . . . , wp +n -\}. Since w is not in a proper parabolic subgroup, we must have wp—Wj > n. 
There exists some s > 1 such that < wp — Wj+ sn < n. By Lemma 14.31 there is some sequence 
/3 = ii<---<ip=j + sn, such that £(wti m> i m+1 ) = £(w) — 1, for each 1 < m < p. Let m be the 
largest index such that i m is a vertex in H\^p. Then the edge e\ = (i m ,i m+ i) is not contained in Hp, 
but is in Gp. 
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Second, suppose r > 2. Let 1 < i < r. By Lemma B~2"1 there exists some integer ai+x < j < ai + \+n 
such that Wj < w ar Then there exists some s > such that < w ai — Wj +sn < n. By Lemma |4.3( 
there is a sequence dj = i\ < ■ ■ ■ < i p = j + sn, such that £(wti m j m+1 ) = £(w) — 1, for each 1 < m < p. 
Let m be the largest index such that i m is a vertex in H, h p. Then the edge = (i m ,i m+ x) is not 
contained in Hp, but is in Gp. Repeat this process for each 1 < % < r. 

Fix j such that Wj = minjw;^, . . . , wp +n -x}. Again, since w is not in a proper parabolic subgroup, 
we have some j3 < k < (3 + n, such that < Wj or > uij+ n by Corollary 12.121 But, by the 
definition of Wj, we cannot have wt < Wj. Hence, Wk > Wj+n- Since w ar is the largest element 
in {wp, . . . , wp +n -i}, we have w ar > Wk > Wj +n . Then there exists some s > 1 such that w ar — 
Wj+sn < ti. So again, by Lemma 14.31 there is a sequence a r = i\ < ■ ■ ■ < i p = j + sn, such that 
£(. w tim,i m +i) = ^{ w ) ~ 1) f° r each 1 < m < p. Let m be the largest index such that i m is a vertex in 
H r ^. Then the edge e r = (i m ,i m+ i) is not contained in Hp, but is in Gp. 

We have found r edges, e%, . . . , e r , which are not contained in Hp. By Lemma [4.41 Hp has at least 
n — r edges. Hence, Gp has at least n edges. □ 

Proof of Theorem \4-l\ Suppose w contains the pattern 4231. Then there exist indices < ii < is < 
il, such that < Wi 2 < Wi 3 < Wi A . If there exists some j < with Wj > Wi A , replace i/± with 
j. Hence, we may assume Wi A > Wj, for all j < i^. By decreasing i\, 12, and is, we may assume 
«2 — ^4 < n an d is — ii < n, and also that there is no %i < j < is with W{ 2 < uij < Wi A and no 
is < j < ii with Wj < Wi 2 . We may then replace 24 by the largest value in the range i*± < j < 12 with 
Wj > Wi 4 . Call the final choice of i^ < ii < ^3 < i\ the normalized pattern. 

Set f3 = ii, and construct Gp as above. By construction a\ = /3 = i^. By normalization a\ < i2 < 
iA+n an d ?2 < a 2 if 0,2 exists. 

By Proposition 12.141 we may assume w is not in a proper parabolic subgroup, and hence c\ = n. 
Note that Lemma 14.51 gives \Ep\ > n. This lower bound on the number of edges in Gp is based on 
the edges in a spanning forest of Hp. So if we can show that there is either a cycle in Hp, or another 
edge, different from the e, found in Lemm a"4"31 that is not contained in any H^g, we will get that 
\Ep\ > n. 

There are three ways the 4231 pattern can appear in w based on how the pattern intersperses 
with ax, 0,2 and 14 + n. 

Case 1: Suppose i\ < a.2 (or i\ < i^ + n in the case where r = 1). 

Then, since 02 — (i\ < n, we have i\ — 24 < n. So, by Lemma 14.31 there is a path in H\$ 
connecting 24 to *2 an d is and a path in H\^p connecting i\ to i2 and is- Hence, there is a 
cycle in H\ p. 

Case 2: Suppose a\ < i2 < is < «2 < i\ (or h < ii + n < i\ if r = 1). 

By the normalization of the 4231 pattern, we have i2 — ii < n and i\ — is < n. By Lemma [4.31 
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there is a path from i% to i% in Gg. Let 7 = (u, v) be the last edge in this path such that u 
is a vertex in Hi g. Also, there is some s > such that < w;j 2 — w;j 1+sri < n. Hence, there 
is a sequence from 12 to %\ + sn. Let 5 = (u', i/) be the last edge in this sequence such that 
v! is a vertex in Hi g. 

li u = u' , then there is a cycle in H\ t g, since there is already a path from £4 to 12 and a 
path from i^ to Z3. If instead, u 7^ u', then j ^ 5. At most one of 7 and 5 can equal ei and 
neither are in Hg. 

Case 3: Finally, suppose a\ < ii < 02 < 23 (or ai < %i < 14, + n < is if r = 1). 

There exists some g > such that < Wj 2 — Wi 1+qn < n. Hence, there is a sequence from 12 
to %i + gn. Let 7 = (u, f) be the last edge in this sequence such that u is an edge in Hi t g. 
Similarly, there exists some s > such that < Wi A — Wi 3+sn < n. Hence, there is a path 
from 14 to + sn. Let 5 = («', u') be the last edge in this sequence such that v! is a vertex 
in H\ g. Since w v < Wi 2 < Wi 3 < w v >, we have 7 7^ 5. At most one of 7 and 5 can equal e\ 
and neither are in Hg. 

□ 



5. Affine Bruhat Pictures 

In this section we introduce an affine version of Bruhat pictures, which first appeared in [4]. We 
use these pictures to flatten a pair x < w as much as possible while preserving the length difference 
and the size of the set 3l(x,w). The key result is Corollary 15.41 

Given an affine permutation w € S n , we can visualize w as {(i, Wi) : i G Z} C 1? . We think of each 
pair (i, W{) as a point in the plane drawn in Cartesian coordinates. Let pt w (i) = (i, W{). Furthermore, 
when comparing two affine permutations x, w in Bruhat order using the rank difference function d x w 
and Theorem l2.9l it is useful to visualize the nonzero entries of the function d XyW as a union of shaded 
rectangles in the plane. Combining both visualizations we get an affine Bruhat picture. 

For example, take w = [8, 3, 1, 0, 4, 5] and x = wti^ = [0, 3, 1, 8, 4, 5], then d x>w will be positive in 
the translated shaded regions of the affine Bruhat picture for x < w in Figure [TJ Here the points of 
w are represented by dots, while the points of x are represented by x's. 

Recall from Section R(x, w) = {t € T : x < xt < w}. Given integers p < q such that x p < x g , 
we will use the affine Bruhat pictures to determine if x < xt Ptq < w. Observe that d XjX t p is positive 
on the periodic union of rectangles 



•A- Pt q t k(x) = [p + kn, q — 1 + kn] x [x p + 1 + kn, x q + kn], for all k G Z. 
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Figure 1. w = [8,3,1,0,5,4] and x = wt 1A = [0,3,1,8,5,4]. 

It is possible that these rectangles overlap in affine Bruhat pictures as in Figure [21 If a point 

is contained in exactly m consecutive translates of A Piq fl(x), then d x ^ x t pq = m. Thus, we get the 

following criterion for determining if t v>q £ Jl(x,w). 

Lemma 5.1. Given affine permutations x < w and integers p < q such that x p < x q , then t Ptq 6 
R(x,w), provided d x ^ w (i,j) > m for every (i,j) contained in m consecutive rectangles A P:q ^(x). 

As an example of computing Jl(x, w) using shading, let w = [6, —3, 8, 5, 4, 1] and x = [1, 2, 6, 5, 4, 3] 
(see Figure [2]) . Then 

0l(x, Ul) = {t±2, il3, t±4, ii5, tiQ, t%3, t24, *25i *26 5 *37j *38i *47i *48^57 5 *58i *67i *68}- 

5.1. Flattening AfRne Permutations. Whenever Xi = u>i, the function d X)W and the Bruhat 
pictures would remain the same if we removed the entire column i and row Xi = W{ from the plane. 
However, removing some of these rows and columns would change £(w) — £(x) and/or w). We 

want to identify which rows and columns where Xi = Wi can be removed while maintaining the length 
difference and #3l(x,w). This will simplify the Bruhat pictures that follow. 

Consider w £ S n as points drawn in the plane. For i £ Z, let w l be the unique affine permutation 
with points obtained from the points of w by deleting columns i + kn and rows Wi + kn for all k G 7L. 
The remaining rows and columns of this infinite array should now be relabeled in such a way that 
w\ + u>?> + . . . + u4_i = (2) • This can be done in a unique way since every window of n — 1 distinct 
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Figure 2. w = [6, -3, 8, 5, 4, 1] and x = [1, 2, 6, 5, 4, 3]. 

values mod n — 1 determines a periodic function from Z to Z with period n — 1 and some translate 
of this window must sum to Q) • 

Lemma 5.2. Suppose x < w € 5 n . Ze£ i 6e an integer such that pt x (i) = pt w (i) and d XjW (pt x (i)) = 
d' x>w (pt x (i)) = 0. Then, £(w) - £(x) = £(w 7 ) - tix 1 ). 

Proof. Recall, £(w) is the number of affine inversions for w. We claim £(w) — £{w l ) = r w (pt w (i)) + 
r' w (pt w (i)) since all the affine inversions which involve a translate of pt w (i) can be represented by a 
point northwest of pt w (i) or southeast of pt w (i). Therefore, 

t{w) - £{x) - {l{w l ) - £(x 7 )) = U(w) - £(w 7 fj - (t(x) - £(x 7 )^ 

= r w (pt w (i)) + r' w (pt w (i)) - (r^pt^t)) + ^(pt^^))) 
= dx tW (pt x (i)) + d' xw (pt x (i)). 

The last line is by assumption, thus proving the lemma. □ 

Lemma 5.3. Suppose x < w S S n . Let i be an integer such that x, = W{ and d(i,Wi + 1) = 
d(i — l,Wi) = 0. Then there is a bisection between Jl(x,w) and Jl(x' l ,w t ). The analogous statement 
holds if, instead, d'(i,Wi — 1) = d'{i + l,Wi) = 0. 

Proof. Consider the rows and columns of w l to be labeled as a subset of the rows and columns of 
w instead of relabeling. Since X{ = Wi, the values of d XjW and d i w ~ agree at all the points where 
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d i -j is defined. Hence, by Lemma [5TTI for every t 0j & € there is a corresponding reflection 

in u>). 

Conversely, we claim every t a ,b £ w) has a ^ i and b ^ i (mod n) and so corresponds 
with a reflection in by Lemma 15. II again. To prove the claim, assume that there exists a 

U,b € R(x,w) with i < b. By hypothesis, d XjW {i,Wi + 1) = 0, so d x t % btW (i,W{ + 1) < 0, which implies 
%U,b ^ w, contradicting the assumption that i^j, € Similarly, since d X)W {i — l,Wi) = 0, no 

ta,i £ it?) exists for a < i. The same proof works if, instead, d'(i, Wi — 1) = d'(i + 1, Wj) = 0. □ 

We say the pair x < w is flattened if no index i exists such that 

(1) Xi = Wi, 

(2) dx iW (pt x (i)) = d' XiW (pt x (i)) = 0, and 

(3) d(i, Wi + 1) = cZ(i — 1, W{) = or cf(i, tUj — 1) = cf(i + 1, wi) = 0. 

Otherwise, the pair is flattenable. Observe that x % < w l since x < w and the values of d XjW and 
d ~ 7 agree at all the points where d 7 7 is defined. Let x < w he the flattened pair obtained from 
x < w by iteratively removing every possible index until the pair is flattened. Then by Lemma 15.21 
and Lemma 15.31 we have the following corollary. 

Corollary 5.4. Suppose x < w G 5 n . T/ien #3?(x,u>) = ^(u>) — if and only if #9l(x,w) = 
£(w)-£(x). 

6. When w Contains 3412 

The goal of this section is to prove that any affine permutation w containing a 3412 pattern is 
either a twisted spiral element or there exists a x < w such that w) > £(w). In the latter case, 

this implies P w (q) is not palindromic and X w is not rationally smooth by Theorem 12.11 We simplify 
the proofs by squeezing the pattern down as much as possible to a normalized 3412 pattern and 
flattening x, w down to the smallest S n possible. The key tool we use is the affine Bruhat pictures 
defined in Section [5j The arguments generalize 0, Section 4] to affine permutations. 

Theorem 6.1. If w 6 S n contains the pattern 3412, but is not a twisted spiral permutation, then 
P w {q) is not palindromic. 

Proof. Suppose w € S n contains 3412 at indices i% < i^ < i\ < %i- By Theorem 14.11 we may 
also assume that w avoids the pattern 4231. We normalize the pattern that occurs by squeezing it 
together as far as possible. This is accomplished using the following procedure. 

(1) Let z 4 < i\ be the maximal index such that there exists some j < z 4 with w^ > Wj > Wi 2 . 

(2) Let i'i > z 4 be the minimal index such that there exists some k > i' x with w^ < w^ < Wj. 

(3) Let i' 3 < i 4 be the maximal index such that Wk < w^ < w^ . 
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(4) Let i' 2 > i'i be the minimal index such that < < w^. 
Keep repeating this process until no more changes occur. Since < ij — i'^ < i\ — i^, this process will 
eventually terminate. Renaming i'j to ij gives a 3412 pattern as in Figure El where the dotted regions 
contains no points in w including the unbounded regions above and below the pattern. Furthermore, 
since w avoids the pattern 4231, any values of w lying in regions A, B, or C must occur in decreasing 
order. 









C 








B 








A 



















h k h h 



Figure 3. Normalization of the 3412 pattern. 

The proof now proceeds by cases depending on whether or not region B is empty and the number 
of congruence classes among {ii,«2>23>H} mod n. Note that i\ = 12 and 23 = 24 are the only possible 
congruences among these indices, since they index a 3412 pattern. 

6.1. When Region B is Nonempty. By the normalization of the 3412 pattern, we must have 
Wi 3 > Wi 4+ i > ■ ■ ■ > w i4+ k > Wi 2 , where k = i\ — 14 — 1 > 1. We will refer to such a pattern as a 
45312^ pattern. Regions A and C must be empty or else w contains a 4231 pattern. Thus, we have 
the picture as in Figure HI where the dotted regions are empty, including the unbounded regions on 
all 4 sides. 




i 3 i A ii i 2 



Figure 4. Normalized 45312 fc pattern. 
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Case 1: All residues are distinct among i2, 13, u}. 

Let x = ^^322^412^3*1 • Based on Corollary 15.41 we can replace x with x and w with w. 
Hence, we may assume that n = k + 4 and that the indices {ii,«2>*3> H, H + lj • • • > *4 + ^} 
are all of the distinct residues mod n. The next step is to consider how the translated values 
of u% , Wi 2 , Wi 3 , u>j 4 , u)j 4+ i , . . . , Wi i+ k intersperse the 45312& pattern. 

We claim that, assuming the 3412 pattern is normalized, the pair x, w is flattened, and 
w does not contain 4231, there are only 4 possible interspersed arrangements as shown in 
Figure [5j Since the pair x, w is flattened, we can assume there are no points in the affine 
Bruhat picture outside the shaded regions. Examining the affine Bruhat pictures in Figure 
we see that, in the first picture, £(w) —£(x) = 2k+3 and #9l(x, w) > 2k+A. In the remaining 
three pictures, we have £(w) — £(x) = 2k + 5 and w) > 2k + 5. Hence, in all of these 

cases, P w (q) cannot be palindromic by Proposition 12.11 

It remains to prove the claim that these are all possible arrangements. If either 12 < 13+n, 
or Wi A < Wi 1+n , then the shaded regions do not overlap, as seen in the first picture of Figure(5j 
In this case, £{w) — l(x) and #Jl(x, w) do not depend on how the other values intersperse 
the pattern. 

Suppose 12 > «3 + n and Wi A > Wi 1+n , so that the translated shaded regions do overlap. 
We must have 12 < %\ + n or else we obtain a dot in one of the forbidden regions in Figured! 
Furthermore, the indices ^4, «4 + l, . . . , 14+k, i\ are all consecutive, which forces 13+11, i2,ii+n 
to also be consecutive. Hence, [wi 3 , Wi 2 - n , W{ 4 , u>i 4 +i, . . . , Wi 4 +fc, w^] is a complete window. 

Observe from Figured! that the values W{ 4+ \, . . . ,Wi 4+ k are all consecutive. Further- 
more, we must also have Wi 2 < Wi 1+n , and Wi A < Wi 3+n , or else we obtain a dot in one of 
the forbidden regions in Figure [H If Wi A+ \ > Wi 2+n , then we may replace the 3412 pat- 
tern with Wj 4+ i_ n t(;j 4+ iU7j 1 u7j2 in the normalization procedure. Thus, the consecutive values 
tt>i 4+ i, . . . ,Wi A+ k must all lie either between w. l2 and Wi 1+n or between Wi 1+n and Wi 2+n . 

Consider first, when Wi 2 < Wi 4+ k < • • • < w-i A +i < Wi 1+n . By our normalization procedure, 
there are two possibilities for Wi A ; either Wi 1+n < Wi 4 < Wi 2+n or Wi A+ \ +n < Wi A < Wi 3+n . 
These cases appear in pictures 2 and 3 of Figure [U In either case, the relative value of Wi 3 
is forced in the interval Wj 4 +i < W{ 3 < w^+n, by the normalization procedure. 

Finally, consider the case where Wi 1+n < wi A+ k < • • • < wi A+ \ < Wi 2+n . If wi A > Wi A+ i +n , 
then Wi A Wi A+ iWi A+ i +n Wi 1+n forms a 4231 pattern. If Wi 2+Tl < Wi A < Wi A+ i +n , then we could 
replace the 3412 pattern with Wi A - n Wi A+ iWi^Wi 2 in the normalization procedure. Hence, we 
must have Wi A < Wi 2+n . This case corresponds to the last picture in Figure Again, the 
relative value of Wi., is forced. 
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Figure 5. Cases with i% =k 24 and %\=^ii (mod n). 



Case 2: Two residues the same. 

There are only two cases to consider; either i% = 14 (mod n) or i\ = 12 (mod n). Note that 
both cannot happen simultaneously, otherwise w^w^+i-nW^+iWi^ gives a 4231 pattern. 
If i 3 = i 4 (mod n), then let x = wti^ti^t^^+n. If instead i\ = (mod n), then let 
x = wti^ti^t^^+n. The picture for the case where is = i$ is given in Figured) The 
picture for the other case can be obtained by turning Figure [6] upside-down. In either case 
we have l(w) - t{x) = 2k + 3 and #K(s, w) = 2k + 4. 

Note that Figure [6] represents the only possible interspersing arrangement in this case. 
In particular, we must have Wi a < Wi 1+n . Otherwise, Wi 1+n < W{ 4+ k by Figure HI and 
w i4 = Wi 3 +n > Wn+i+n, and hence Wi 4 w ii+1 Wi 4+ i +n w il+n gives a 4231 pattern. Similarly, if 
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i\ = %2 (mod n), then we must have Wi A < Wi 2+n , or else Wi A Wi A+ iWi A+ i +n Wi 2 gives a 4231 
pattern. 




Figure 6. Case with i$ = 14 (mod n). 

6.2. When Region B is Empty. By the normalization in Figure El we may assume i% = + 1 
since region B is empty. 

Case 1: Either 13 =j£ ^4 or i\ ^ 22 (mod n). 

In this case we do not flatten the affine permutation and use Bruhat pictures. Instead, we use 
the methods and notation from Section [J] to show directly that the Poincare polynomial fails 
to be palindromic at degree 1. Fix /3 such that wp = m.axj<i 3 {wj}. Hence, (3 < 13 < f3 + n. 
Construct the graph Gp, as in SectionJU and let e±, . . . , e r be the extra edges from Lemma l4"31 
By Proposition 12.141 we may assume w is not contained in a proper parabolic subgroup. 
Hence, we need only show that |-E^| > n. By Lemma 14.51 \Ep\ > n so we need only show 
there exists one edge not in Hp or in {e\, . . . ,e r }. Suppose 23 is a vertex in H^p C Gp. 
There are three ways the 3412 pattern can occur in Gp in this case, 
(a) «4 < 13 + n. 

If %2 — 13 < n, then 7 = (13,12) is an edge in Gp, by Proposition 12.81 Otherwise, there 
exists some m > such that < W{ 3 — Wi 2 + mn < n, in which case 7 = (^3,^2 + rnn) 
is an edge in Gp. Since i\ = i± + 1 and 14 < 13 + n, we also have i\ < 13 + n. Hence, 
there is a sequence 13 = j\ < J2 < • • • < j p = i\, such that £(wtj m j m+1 ) = i(w) — 1 for 
all 1 < m < p, by Lemma 14.31 which gives us a path in Gp from 13 to i\. Let 5 be the 
first edge in this path not contained entirely in H^ p. Since 13 < 24 < %\ and Wi A > u>i 3 , 
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is and i\ cannot both be in H^p, we conclude such an edge 5 exists. At most one of 7 
and S can be the added edge and neither edge is in Hp. 

(b) 14 > 13 + n and ii ^ 22 (mod n). 

There exist u, v > such that < t0j 3 — f%+ W 7i < n and < Wi 3 — Wi 2+vn < n. 
By the way we have normalized the 3412 pattern, there is no 13 < j < i% + to with 
Wi 2+vn < Wj < Wi 3 , since Wj- vn cannot be in the dotted region of Figure El Hence, 
7 = (^3^2 + vn) is an edge in Gp. 

To identify a second edge in Gp, we must consider the different ways pt w (ji + un) and 
pt w (i2 + to) are arranged. If pt^^i + un) is southwest or northeast of pt to (*2 + to), 
then let 5 be the first edge in the chain from 13 to i\ + un found in Lemma 14.31 that 
is not contained in Hi g. Then 7 ^ i, since their right endpoints are guaranteed to be 
different. At most one of 7 and 8 can be the added edge and neither edge is in Hp. 

Suppose instead, pt w (i\ + un) is southeast of pi w (t2 + to). Then i\ + un > 12 + vn 
and Wi 1+un < Wi 2+vn and hence i\ > 12 + (v — u)n and < w i2+ ^ v _ u ^ n . Recall we 
have normalized our permutation so that either 12 — it < n or Wi 2 — < n. But the 
southeastern configuration can only happen if *2 — *i < n - Hence, 13 < 12—n < 24 = 13+n. 

If 12 — n is not a vertex in H^p , then let 5 be the first edge in the sequence from 13 to 
12 — n that leaves H^^p. Otherwise, if 12 — n is a vertex in then let 5 be the first 

edge in the sequence from 12 — n to i\ that leaves Hi^p. In either case, 5 is different from 
7, since its right endpoint is smaller. At most one of 7 and S can be the added edge 
and neither edge is in Hp. 

(c) ?4 > 13 + n and ii = 12 (mod n). 

As in Case llbl there exists some s > 1 such that < Wi 3 — Wi 1+sn < n and the edge 
7 = (^3^1 + sn) is in Gp. Since wi A > w; L2 = Wi 1+n , we have Wi 4 - n > w^. Since 
14,-13 > n, we must have Wi 4 — Wi 3 < n. In fact, by the normalization of the 3412 
pattern, this forces < Wi A - n < w; l2 . There are two cases to consider. 

If 14 — n is not a vertex in Hi p, then let 5 be the first edge in the sequence from 13 to 
14 — n that leaves Hi t p. If instead, 14 — n is a vertex in Hi^p, then let 5 be the first edge 
in the sequence from 14 — n to i\ that leaves H^g. In either case, 5 if different from 7, 
since the right end point of 5 is smaller than Wi 2 . At most one of 7 and 5 can be the 
added edge ei and neither edge is in Hp. 
Case 2: Every normalized 3412 pattern has 13 = 14 and i\ = 12 (mod n). 

We can assume that 13 = 14 — n and Z2 = h + n. Then w is as in Figure [7] where region £J 
is the rectangle with corners at {i\ — n,Wi 4 -2 n ) and (i4,u>j 1+n ). Since w avoids 4231, either 
all of the values in region E are decreasing or E is empty. 
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Figure 7. A normalized 3412 pattern with only two distinct residues. 

(a) Region E is nonempty. 

There exists some ii—n<j < 14 with < Wj < w^+n. By the normalization process, 
Wi 4 - n < Wj +n < Wi 1+ 2n, since there is no i\ < k < i\ + n with Wi 1+n < Wk < Wi A - n . 
Hence, w ii+rn < w il+ ( r+1 ) n < w ii+ ^ r _i) n for all reZ. Let x = wt^-n^t^^. Replace 
x, w, with the flattened pair x, w. Any indices with values in regions D or F, aside 
from £4 and i\, are removed during the flattening process, and we are left with the 
picture in Figure [HJ Suppose there are k > 1 (decreasing) indices in region E. Then 
£(w) - £(x) = 2k + 2 and w) = 4k + 2. 

(b) Region E is empty, but regions D and F are both nonempty. 

We can assume that pt^^ — 1) is in F and pt w (ii + 1) is in region D + n since every 
normalized 3412 pattern has only two congruence classes among &i,i2>*3>ii and E is 
empty. Let x = wti 4 ^ n -i t i 1+ iti 4 -ij 1 +iti 4 - nt i 1 ti 4 i 1 . Replace x,w, with the flattened pair 
x, w, so that we have flattened onto the consecutive indices {14 — 1, 14,11,11 + 1}. Fix 
A; > 3, such that < Wi 4 — Wi 1+ f cn < n. Figure [9] shows the picture when k = 3 and 
k = 4. Regardless of k, we have £(u>) — i{x) = 8 and #R(x, w) = 12. 
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Figure 8. Region E is nonempty. 




FIGURE 9. Region E is empty, but regions D and F are nonempty (k = 3,4). 

(c) Region E is empty, and exactly one of regions D and F is empty. 

Note, one of them must be nonempty whenever n > 3. Assume, D is nonempty, the 
other case being similar. 



RATIONALLY SMOOTH AFFINE SCHUBERT VARIETIES 



29 




Figure 10. Region E is empty and region D has an ascent (k=2). 

If there is an ascent among the values in region D, choose a leftmost pair i\ < a,b < 
Z4 + n such that w a < Wb- For example, one could let a > i\ be the smallest index such 
that there exists a a < j < i\ + n with w a < Wj < w% x , and let b > a be the smallest 
index such that w a < Then w^-nW^w^WaWi, forms a 45312^ pattern. Following the 
same reasoning as in Case 2 on Page[23l let x = wti 4ta t at btb,i 4 +n and replace x,w by the 
corresponding flattened pair. The affine Bruhat picture for k = 2 is shown in Figure [TUl 

For all k > 1, we have £(«;) — £(x) = 2k + 3 and ^3l(x,w) = 2k + 4. The picture for 
the case F is nonempty and contains an ascent can be obtained by turning Figure [TOl 
upside-down. 

Finally, if the values in region D are all decreasing, then the entries must be n — 1 
consecutive values in consecutive positions, or else either region D or E will be nonempty 
since w is a bijection. In this case, we claim w is a twisted spiral permutation. 

To prove the claim, let k > 2 be the integer such that Wi 1+ k n = Wi A — 1. Then 
w ■ c(i4,k(n — 1)) = wo, using the notation from Section [231 Hence, by (|2.3p . w is a 
twisted spiral permutation. 



□ 
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7. Further Directions 

Unlike rationally smooth Schubert varieties, there are only finitely many smooth Schubert varieties 
corresponding to affine permutations. In the proof of Theorem 13.11 we saw that any 3412 and 4231 
avoiding affine permutation can be written in the form w = w'a, with £(w) = l(w') + £{a). Both 
w' and a are elements of a proper parabolic subgroup of S n , and hence £(w'),£(a) < Q). Thus, we 
have the following corollary. 

Corollary 7.1 (To Theorem EOJ . If w € S n avoids 3412 and 4231, then i(w) < 2("). 

Since the number of affine permutations of length at most 2(!J) is finite, we would like to compute 
how many affine permutations in S n avoid both 3412 and 4231. Conjecturally, this is equivalent to 
the number of smooth affine Schubert varieties of type A n . Starting with n = 2, the first few terms 
of this sequence are 5,31, 173,891,4373, which did not previously appear in Sloane's 

In Q| , Bjorner and Ekedahl give general inequalities amongst the coefficients of the Poincare 
polynomial for elements of any crystallographic Coxeter group. In Theorem 14. 1\ we prove n = c\ < 
c e(w)-i- Combining this fact with [6, Theorem C], proves the following corollary. 

Corollary 7.2. Let w € S n and assume w contains a 4231. Then if Pid, w = 1 + Q>\q + ■ ■ ■ + adq d is 
the Kazhdan-Lusztig polynomial indexed by id, w, then a\ > 0. 
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